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Executive Summary 


Since 2010, CPS has focused resources on schools in order to improve school climate — 
particularly school safety. These resources were used, to a large extent, to hire vendors 
that provided services to teachers and students in 47 high schools in Chicago. 

This memo will provide information to CPS about the perspectives of principals, 
coordinators, vendors, students, and teachers in the Culture of Calm schools. Data from a 
web survey of principals and coordinators, focus groups of vendors, and student and 
teacher surveys were analyzed to examine possible effects of the initiative. 

Coordinators, principals, and vendors generally perceived their efforts in schools as 
positive. Most reported that, as a result of the initiative, school climate improved. 

Although student reports of safety did not improve at a faster rate in Culture of Calm 
schools compared to other similar high schools, teachers’ reports of a safe environment, 
marked by less crime and disorder improved at a significantly faster rate than in other 
schools. 

Student reports of academic personalism increased dramatically between 2009 and 2011 
for Culture of Calm schools compared to non-Culture of Calm schools. Few other indicators 
of school climate improved substantially for Culture of Calm schools compared to other 
high schools. 


The Culture of Calm Initiative 


In late 2010, Chicago Public Schools initiated a new safety program called the Culture of 
Calm initiative (CoC). The goal of CoC was to improve school climate by improving the 
following within schools: 

• Leadership and staff commitment; 

• Behavior frameworks; 

• Staff development; 

• Student development; 

• Community engagement; and 

• Performance management. 

Forty seven high schools were selected by CPS to participate in CoC. Most participating 
schools were neighborhood schools; only two were vocational. Twenty nine were 
predominantly African American, 11 were predominantly Hispanic, and seven were mixed 
minority. As Table 1 shows, most schools (94%) were located in what are now the south, 
west, or southwest side networks. 


Table 1. Number of schools participating by network. 


Network name 

ft schools in CoC 

Total ft schools 
in network 

Far South Side 

7 

12 

North/Northwest 

3 

27 

South Side 

14 

30 

Southwest Side 

10 

27 

West Side 

13 

46 


Compared to students in neighborhood schools not in the CoC initiative, the students 
served by the CoC schools came from neighborhoods with higher adult unemployment, 
with more families living below the poverty line, and with higher crime rates. In addition, 
they had slightly lower incoming test scores, were more likely to be receiving special 
education services, and were more likely to be receiving free or reduced price lunch. 

The initiative was intended to work through a combination of mechanisms. As one part of 
the initiative CoC schools were provided funds that could be spent on professional 
development for teachers, mentoring for students, case management, and curriculum. 
Some activities directly involved school staff and changes in school practices or policies. 
For other programs schools hired vendors, often community organizations, to deliver 
services. Most schools had a designated CoC coordinator to oversee the efforts of both the 
school and its service providers. 




Six of the 47 schools (Clemente, Farragut, Harlan, Julian, Manley, and Robeson) were 
selected as pilot schools. These schools received more financial resources as well as 
consulting support from the Central Office. 

The questions to be addressed in this memo are: 

1. How was the initiative implemented? What strategies were used, and what 
roadblocks existed in implementing the strategies? 

2. What were the perceived effects of the initiative from the perspective of 
coordinators, principals, vendors, 

3. Have student and teacher perceptions about their school’s climate changed more 
since the introduction of CoC than the perceptions of similar students in similar 
non-CoC schools? 

To answer the first two questions we rely on data collected in summer 2011 through 
surveys of CoC principals and coordinators and focus groups of vendors. To answer the 
third question, we used data collected as part of the district-wide My Voice, My School 
survey, administered in spring 2011. More details on the methods can be found in 
Appendix A. 

Implementation of CoC: Collaboration, Integration, Strategies and 
Barriers 

With CCSR’s support, CPS administered an online survey to principals and coordinators, 
receiving responses from 30 of 46 principals (65%) and 31 of 45 coordinators (69%). We 
asked them to describe the activities and programs directly undertaken by school staff to 
implement CoC as well as the strategies and programs provided by outside vendors. We 
asked about how schools and vendors worked together; we also asked about perceived 
barriers to implementation and perceived signs of success. 

CCSR conducted two focus groups attended by 14 representatives of 12 vendors, all of 
whom worked in more than 1 school. We asked approximately the same questions. See 
Appendix A for more details on the methodology. 

Collaboration 

According to principals and coordinators, a wide range of school staff worked on vendor 
programs, including principals and coordinators themselves, but also including teachers 
and other staff. At only three schools did a principal or coordinator report that vendors 
worked only among themselves. Vendors also reported working closely with principals, 
assistant principals, department heads, deans, and coordinators on the strategies. Vendors 
often — but not always — reported having a positive relationship with these individuals. 
Principals and coordinators were seen as crucial partners in the Culture of Calm. One 
vendor described the initiative as a "seamless process” when the principal was committed. 


But in one school, a principal who was leaving his position after the school year walked out 
of a training session and told teachers they didn’t have to participate. Without this 
principal’s buy-in, participation and support from teachers dropped precipitously. Vendors 
repeatedly mentioned that the principal needs to be held accountable for the success of 
program implementation. 

The vendors found that having an ally within the school — whether a principal, coordinator, 
or a teacher — was important. In one school, the dean helped the vendor understand what 
the school’s needs were; he also found rooms for the vendors as needed. 

Participants in the strategies: Students and teachers entered programs in a number of 
different ways. In some programs, teachers and students volunteered to participate. In 
these schools, vendors marketed their programs in various ways — by word of mouth or 
advertisements on the walls. 

In other programs, principals and/or coordinators nominated students for the program. 
This nomination procedure had mixed results. One program, for example, targets girls who 
are having problems in school. The vendor for this program reported that the recruited 
students were all honors students. The vendor had to recruit other students who were 
eligible for the program, which reduced the time they could spend delivering services. 

Some vendors expressed a wish that they would have more visibility early in the program. 
One vendor believed that an assembly would have brought attention to his program, 
making recruiting easier. Others agreed that many students and teachers were unaware of 
their programs, even after several months in the schools. 

Vendors worked with principals, assistant principals, department heads, deans, and 
coordinators on the strategies. School administrators were sometimes active in the 
selection of students or coordination among different programs. 

Integration 

Twenty-two of the coordinators indicated that vendor programs were either somewhat or 
tightly connected to teaching and learning at the school, as did 19 principals. In contrast, 
over 90% coordinators and principals responded that the school-based programs were 
either loosely connected to teaching and learning or operated independently. 

Strategies 

Strategies mentioned by principals/coordinators: On the principal/coordinator survey, we 
provided a list of possible school-based and vendor-based strategies based on 
conversations with central office staff. We asked respondents to identify as many as they 
felt were primary or secondary in their school. We also provided space for respondents to 
type in strategies not on the list. 


As groups of respondents replying to our suggested list of strategies, both principals and 
coordinators agreed on the ranking of the following as the primary school-based strategies 
at their schools: discipline policies, attendance policies, providing incentives for students to 
modify their behavior, involving parents, maintaining accurate records, redeploying 
security staff, making use of advisory period, and reconfiguring staff. They also generally 
agreed on identifying the primary strategies used by vendors, with anger management and 
mentoring topping the list. PD for teachers around classroom management issues, peer 
mediation, and tutoring were in positions 3 through 5 for principals, with leadership 
development for students being cited least often; more coordinators perceived leadership 
development as primary, with slightly fewer citing PD, peer mentoring, and tutoring. 
However, this list is obviously not exhaustive. When we provided space for respondents to 
provide additional strategies used by their schools, we received multiple additional 
strategies from over two thirds of coordinators and over one third of principals. Many of 
these additional suggestions were particular instances of our initial list; others were not. 
We grouped the whole list into nine broad categories. 1 Here we report the number of 
schools using each broad category as reported by coordinators. Principal results are similar 
but less detailed. 

Table 2 details the strategies used across schools. It includes those things mentioned either 
as a primary or secondary program focus, whether it was school-based or vendor-provided. 
All coordinators reported using at least one of these strategies in their schools. 

Table 2. Number of schools implementing strategies. 


Strategy group 

# schools 

Academic Support 

26 

Shared understanding of Rules 

4 

Attendance and Discipline Policies 

30 

Extracurricular Programs 

3 

Improving Student Involvement in 
School 

29 

Outreach to family or community 

30 

Reorganizing Personnel 

25 

Building Social Skills: Students 

31 

Skills Building: Teachers 

26 


Strategies mentioned by vendors: Vendors reported providing services related to mentoring, 
professional development for teachers, self-regulation programs for students, the arts, help 
with substance abuse issues, and counseling. One vendor also reported using what he 
called a "covenant" with students — a promise that they would go to school and get good 
grades. Students who kept their promise would be rewarded with an incentive (camera, 
watch, or cash]. Another vendor reported using a newsletter to highlight students’ progress. 


1 Some of the less-obvious strategies under attendance and discipline policies included changing the focus of 








Many vendors worked in multiple schools. Although their basic programs stayed the same, 
nearly all vendors reported making minor adjustments for student, teacher, and principal 
needs. For example, some vendors described having difficulty getting students to come to 
their programs in some schools but not others. In those schools where attendance was a 
problem, vendors provided food as an incentive. 

Barriers 

Barriers reported by coordinators/principals: Coordinators’ reports of barriers were 
generally consistent. 21 coordinators reported that time was an issue. Within this broad 
category, respondents were asked to specify all barriers to implementation of CoC 
strategies that they experienced; coordinators in 18 schools reported that staff time was a 
barrier; four reported that the length of time to implement the strategies was a barrier (for 
example, programs being implemented later in the school year); one coordinator reported 
that there simply were not enough hours in the school day to implement programs. 

Student attendance in programs was also a significant issue for coordinators. This was also 
discussed in the vendor focus groups, which will be discussed below. 


Table 3. Number of schools experiencing barriers. 


Barriers 

# schools 

Buy-In 

14 

Resources 

2 

Staff problems 

6 

Student attendance 

20 

Time 

21 

Too many programs going on at once 

0 

Vendor staff problems 

11 


Barriers mentioned by vendors: Vendors agreed on several main barriers. These included 
barriers at the school level and at the district level as well as buy-in from teachers, students, 
and parents. Barriers to implementation were: 

• Difficulties with statements of work and delays in getting started 

• Limitations on the kinds of programs they could provide 

• Lack of support from the principal 

• Lack of support from the coordinator or no coordinator at all 

• No space available in the schools 

A consistent barrier to implementation across vendors was gaining approval and funding 
from CPS central office. Statements of Work (SOW) were generally seen as onerous, 
stressful to complete, and difficult to submit. 




Most vendors reported some difficulties in developing a budget and difficulty negotiating 
the line items with central office. Budgets were revised after consulting with CPS many 
times. 

Feedback on the SOW was often slow. Most vendors received late approval and therefore 
had a short timeframe to work with teachers and students. For example, one vendor 
reported that her program is meant to take 15 weeks, but CPS approval was slow, cutting 
the program’s implementation time to 9 weeks. Another reported only having three 
months to work with students. One vendor expressed her frustration: "It appeared as if 
someone didn’t want it to work. Any time you’re given a mandate and nine months to get it 
done. ..and vendors can’t start until three months from the end, someone doesn’t want it to 
work." 

CPS requirements were also seen as restrictive. One vendor reported that her program was 
meant to include sports or recreation, but CPS did not approve that aspect of the work. 
Another vendor reported that she intended to take the students in her program on a day 
trip, but the provisions of her contract prohibited her from buying food for the students. 
Reporting requirements were often seen as onerous. One vendor reporting "scrambling" to 
finish a long report required by her contract, but she never received any feedback. 

Coordinators have a unique role in the Culture of Calm as an ally and manager of the 
different programs in the school. Vendors reported working with some coordinators 
closely. In this case, coordinators held meetings with all of the vendors working in the 
school in order to coordinate the various programs. These vendors might include Culture 
of Calm vendors as well as other program providers working in the school. 

In other schools, vendors were not working with — or aware of — other vendors. This led to 
a duplication of efforts and a drop in overall attendance across vendors. 

Other coordinators were less involved in the initiative, particularly when they had other 
tasks. One vendor described difficulty with getting access to data from coordinators who 
were too busy with other responsibilities. This vendor eventually collected her own data. 

Plans for the coming year. All of the vendors wanted to remain working in the same schools 
in the upcoming school year. Most described a disappointment at having to leave the 
students "without a goodbye." They felt that the stability of the program in the school 
would be helpful at improving the culture at the school. One vendor remarked, "It’s so hard 
to get teenagers to open up. If the program returns, they may not trust it." 

Most vendors in the focus groups were unsure what the funding and approval situation 
might be. However, every vendor mentioned that the 2010-2011 school year was a learning 
experience. They felt that they learned how to navigate the CPS bureaucracy, how to 
improve student and staff buy-in, and how to work with principals, coordinators, and other 
vendors. 



Vendor perceptions and school staff perceptions of strategies and barriers 

Vendors, coordinators, and principals all seemed to agree that time was a serious issue. 
Although lack of school staff time was a more frequently reported barrier for coordinators 
and principals than vendors, having limited time in the school year or in the school day was 
reported as a major problem for all parties. In addition, although delays due to late 
approvals of Statements of Work were more salient for vendors, some coordinators and 
principals agreed that this was a barrier. 

Student attendance was also seen as a problem by all parties. However, vendors differed 
from coordinators and principals in attributing the cause of low attendance. Vendors 
reported problems with school staff. One vendor noted that the principal permitted her 
program only after school hours. Another mentioned that school staff generally did not 
buy into the program or was unaware of his program; as a result, students were not 
referred to the vendor for services. 

Coordinators and principals, however, attributed low student attendance as a failure of the 
vendor staff — a "lack of motivation (on their end) to step up student recruitment efforts” or, 
generally, "vendors not consistently reaching out to students." 


Perceived effects of Culture of Calm 


Effectiveness as described by coordinators and principals: We asked coordinators and 
principals to report on the efficacy of vendors’ efforts and the efficacy of nonvendor 
strategies (see Table 4 for question text). In general, coordinators and principals both felt 
that vendors’ efforts were successful and that other efforts that did not use vendors were 
also successful. 

Table 4. Perceptions of efficacy among coordinators and principals. 

Coordinators Principals Total 

Thinking about the strategies used that involved vendors, how effective did you think they 
were at increasing school safety? 


Very ineffective 

6 

3 

9 

Somewhat 

1 

2 

3 

ineffective 

14 

10 

24 

Somewhat effective 

9 

9 

18 

Very effective 

Total 

30 

24 

54 


Thinking about the strategies specified above that did not use vendors . how effective did you 
think they were at increasing school safety? 


Very ineffective 

6 

3 

9 

Somewhat 

2 

2 

4 

ineffective 

11 

5 

16 

Somewhat effective 

7 

11 

18 

Very effective 

Total 

26 

21 

47 


Effectiveness as described by vendors: In general, focus group participants felt good about the work they 
did. All of them believed their work helped students in some way. However, metrics were not always 
easily accessible. One vendor tracked her students’ grades and attendance because the coordinator would 
not release that information. 

Change in Student and Teacher Perspective over Time 

CCSR has surveyed students in grades 6-12 and all teachers on a biennial basis for over 20 
years. The survey, most recently called My Voice, My School, measures constructs found to 
be closely linked to increases in student learning. In particular, the survey provides a 
picture of a school’s climate and its ability to organize for improvement. We wanted to see 
if there were changes in these indicators over time for schools that were part of the 
initiative in comparison to those who were not. Since some CoC schools were not in 
existence prior to the 2004/2005 school year, we analyzed data from the four survey 
administrations between 2005 and 2011. See Appendix A for more details on the survey 
methods. 

Table 5 below shows the survey measures that we believe would be most strongly related 
to the initiative. 


Table 5. My Voice, My School survey measures examined. 


Measure name 

Description 

Student measures 

Safety 

How safe students feel both in and around the school 
building, and while they travel to and from home. 

Academic Personalism 

How much teachers connect with students in the 
classroom and support them in achieving academic 
goals. 

Student-Teacher Trust 

How much students and teachers share mutual trust 
and respect. 

Teacher measures 

Disorder and Crime 

How much disorder teachers report in the hallways, 
physical conflict among students, vandalism, robbery 
or theft, and threats of violence against teachers. 

Program Coherence 

To what extent programs are coordinated and 
consistent with stated goals for student learning. 

Teacher-Parent Trust 

To what extent teachers view parents as partners in 





improving student learning. 

Quality Professional Development 

To what extent professional development is rigorous 
and focused on student learning. 

Teacher-Principal Trust 

How much teachers and principals trust and respect 
one another. 


We used Hierarchical Linear Modeling (HLM) to examine relative changes in student and 
teacher measures over time for CoC schools compared to non-CoC schools. As discussed in 
Appendix A, Rasch analysis was used to estimate a summary score for each measure. 

Comparison schools included all high school and vocational high schools in existence in 
2011 (total of 38 schools). Although CoC schools were selected because they were different 
from other schools, we controlled for the above variables to make the two groups look 
similar to one another. 

For student measures, we controlled for individual students’ race, gender, grade, special 
education status, concentrated poverty in the students’ neighborhoods, social class in the 
students’ neighborhoods, average crime in students’ neighborhoods; we also controlled for 
school characteristics including number of students enrolled in the school, average 
incoming achievement level, and school-level nonresponse. For teachers, less information 
was available because the survey has always been anonymous; teacher data could not be 
linked to auxiliary data like personnel files. Analyses of teacher measures controlled for 
school characteristics only: racial composition of the students, proportion of students 
receiving free or reduced lunch, proportion of special education students, average crime in 
students’ neighborhoods, average concentrated poverty in the students’ neighborhoods, 
average social class in the students’ neighborhoods, number of students enrolled in the 
school, average incoming achievement level, and school-level nonresponse. 


Table 6. Control variable averages in Culture of Calm and non-Culture of Calm 
schools. 


Control Variable 

Non-CoC 

schools 

(n=38) 

CoC 

schools 

(n=47) 

% schools predominantly African American (70%+) 

32% 

64% 

% schools predominantly Hispanic (70%+) 

24% 

23% 

% schools mixed minority (70% African American and 
Hispanic; neither group greater than 70%) 

42% 

13% 

% schools integrated (>30% white) 

3% 

0% 

Crime in students' neighborhoods 2 

-2.03 

-1.70 

Incoming achievement: reading 3 

235.26 

232.42 


2 Crime was calculated as the log of the total number of crimes within a student's census block neighborhood. 
This log(total crime) was averaged within each school. Higher values of crime mean more incidents. 

3 9th graders in each survey year were linked to their 8 th grade ISAT scores. These scores were averaged to 
the school level. Both incoming achievement scores (reading and math) were calculated in this way. 










Incoming achievement: math 

253.99 

251.66 

% students with special education status 

12% 

16% 

% students receiving free or reduced lunch 

88% 

93% 

Concentrated poverty in students’ neighborhoods 4 

-0.02 

0.32 

Social class in students’ neighborhoods 5 

0.00 

-0.20 

% students responding to 2011 survey 

78% 

69% 

Number of students 

892.26 

795.34 


Did Culture of Calm schools improve from 2009 to 2011, relative to non-Culture of Calm 
schools? 

Because the program was implemented in 2010, we focus on the change from 2009 to 2011. 
However, we have included 2005 and 2007 as a way to provide context of the trends. 

Student perceptions of Safety. Students were asked whether they felt safe in and around 
their school and getting to and from school. This measure was, overall, lower in CoC 
schools compared to non-CoC schools. This was the case across all years and is not 
unexpected; CoC schools were selected, in part, because of higher numbers of serious 
incidents. Between 2009 and 2011, both CoC and non-CoC schools improved in Safety — 
but at the same rate. See Figure 1 for the trends since 2005, controlling for the variables 
specified above. 


4 Concentrated poverty is calculated from Census data from students’ Census block neighborhoods 
(specifically the percent of adult males employed and the percent of families with incomes above the poverty 
line), and is standardized such that a "0” value is the mean value for census block groups in Chicago. 

5 Social class is similarly calculated from Census data from students’ Census block neighborhoods. The 
Census variables used for this measure are mean level of education of adults and the percentage of employed 
persons who work as managers or professionals. 





Figure 1. Safety for Culture of Calm and non-Culture of Calm schools. 


Safety 



Non-Culture of Calm 

Culture of Calm 

- — — -All CoC-eligible high 
schools 


Teachers' reports of Disorder and Crime. Teachers were asked to report on such things as 
disorder in the hallways, threats of violence to themselves, and fights among students. This 
measure decreased at a faster rate for CoC schools compared to non-CoC schools. Figure 2 
displays trend lines 6 for CoC schools, non-CoC schools, and a reference line for all high 
schools. As expected, CoC schools have been higher than non-CoC schools on Disorder and 
Crime over the years. However, unlike the student Safety measure, between 2009 and 
2011, the decrease for CoC schools is greater than non-CoC schools. The red line has a 
sharper drop between 2009 and 2011 than the blue line. This effect is moderately strong 7 , 
but we believe it is substantively important. All of the Disorder and Crime questions ask 
about the extent of the problems inside the school, including physical conflicts among 
students, robbery or theft, gang activity, disorder in classrooms, disorder in hallways, 
student disrespect of teachers, and threats of violence toward teachers. CoC programs 
aimed to reduce these incidents directly. 

In contrast, the student measure of Safety — which did not change significantly between 
2009 and 2011 for CoC schools compared to non-CoC schools — asked four questions about 
how safe students feel: in the hallways and bathrooms of the school, outside around the 
school, traveling between home and school, and in classes. Although two of these items ask 
about safety inside the school, the other two do not. We might expect improvements in 
students’ reports of Safety in the longer term; however, the reduction in Disorder and 
Crime reported by teachers is encouraging. 


6 Disorder and Crime was not asked prior to 2007. 

7 Cohen’s d is a measure of the effect size. Typically, when d<0.1, the effect is considered trivial; when d>0.1 
and d< 0.3, the effect is considered small; when d>0.3 and d<0.5, the effect is considered moderate; and when 
d> 0.5, the effect is considered large. For Disorder and Crime, the effect size of being a CoC school on the 
change in this measure between 2009 and 2011 is d=0.4. 


Figure 2. Disorder and Crime for Culture of Calm and non-Culture of Calm schools. 



Students’ reports of Academic Personalism. This measure asked students to report on 
whether their teachers paid attention to their learning and were willing to offer support 
when needed. This measure improved for CoC schools between 2009 and 2011 compared 
to non-CoC schools. We expected this measure to improve because perceived support from 
teachers might increase as a result of the attention paid to student safety. See Figure 3 for 
the trends. This effect is relatively large 8 — however, schools are quite similar to one 
another in Academic Personalism. Any small change in this measure, therefore, seems 
substantial statistically but may not be substantial substantively. 


8 d=l.l 


Figure 3. Academic Personalism for Culture of Calm and non-Culture of Calm schools. 



Teachers' reports of Teacher-Principal Trust. Teachers were asked whether their principal 
has confidence in teachers and whether they trusted him/her. It was expected to improve 
at a faster rate for CoC schools compared to non-CoC schools because the goal of CoC was to 
provide resources for principals to improve school climate. Perhaps teachers in CoC 
schools would trust their principals more as they work to improve school safety. As shown 
in Figure 4, there is an improvement for CoC schools relative to non-CoC schools between 
2009 and 2011; however, this difference in slopes is only marginally significant. 

Figure 4. Teacher-Principal Trust for Culture of Calm and non-Culture of Calm 
schools. 



All other measures tested (Student-Teacher Trust, Program Coherence, Quality 
Professional Development, and Teacher-Parent Trust) did not change significantly between 
2009 and 2011 for CoC schools compared to non-CoC schools. Trend lines are included in 
Appendix C. 

Could these results be explained by any changes across the district? 

Although there are improvements in CoC schools compared to non-CoC schools for some 
measures, others — particularly student perceptions of Safety — do not improve. However, 
other initiatives may be affecting all high schools equally. It is possible, for example, that 
attention paid to high schools in the district after Derrion Albert’s death may have 
improved safety throughout all high schools because of increased attention paid to safety 
to high schools throughout Chicago. In addition, CPS provides many different programs 
across the district, preventing us from pinpointing the exact impact of specific initiatives 
like CoC. We wanted to see whether any of these trends were specific to high schools only. 

The following analyses show that high schools did improve relative to elementary schools 
on a number of different measures — including many that did not differ between CoC and 
non-CoC in the change from 2009 to 2011. Note that only vocational and neighborhood 
schools are included in this analysis. In addition, incoming achievement was dropped from 
the model. See Appendix C for the measures trends for elementary schools compared to 
high schools 

Safety improved district-wide for high schools relative to elementary schools in 2009. In 
fact. Safety was flat for elementary schools between 2009 and 2011, while high schools 
improved. In 2011, Safety in high schools is just slightly lower than elementary schools. 
The fact that CoC schools did not improve relative to others could be that the others were 
already improving at a steep rate. 

Academic Personalism has also been improving district-wide. However, elementary 
schools improved at a faster rate between 2009 and 2011 than high schools. Teacher- 
Principal Trust has also shown greater improvement for high schools compared to 
elementary schools. 

Student-Teacher Trust, Program Coherence, and Quality Professional Development have 
been increasing between 2009 and 2011 at the same rate for elementary schools compared 
to high schools. Similarly, Disorder and Crime has dropped substantially for all schools in 
the district at the same rate between 2009 and 2011. 

Did pilot schools improve at a faster rate than non-pilot CoC schools? 

Culture of Calm was implemented in a slightly different fashion for six of its schools. These 
pilot schools received more money and support from CPS to implement CoC. We repeated 
the same analysis as described on page 11; this time, we restricted the sample to CoC 
schools and examined the impact of being a pilot school compared to being a non-pilot CoC 
school. An important limitation of this method is that the sample size is quite small — only 


six schools — and the reference group consists of the other 41 schools. We anticipated the 
possibility that this analysis would provide null results because of this small sample size. 
Our suspicions were confirmed: pilot schools did not differ in the change in any measure 
between 2009 and 2011 from non-pilot CoC schools. Appendix C contains each of the trend 
lines. 


Did pilot schools improve at a faster rate than non-CoC schools? 

Although pilot schools were not significantly different from non-pilot CoC schools, we 
thought we might detect some differences between pilot schools and non-CoC schools. 
However, we did not find any significant differences in the change of any measure between 
2009 and 2011 for pilot schools compared to non-CoC schools. As noted previously, this 
may be due to a small number of pilot schools. 


Appendix A. Methodology. 


Web survey of principals and coordinators 

The survey was conducted in June 2011. The survey was implemented online using the 
Surveymonkey application. No incentives were provided. Principals and coordinators 
were contacted via their CPS email accounts and invited to participate. If they did not 
respond, they were sent email reminders after one, two, and three weeks. 

Although 47 schools participated in the initiative, one school did not have a principal and 
three schools did not have a coordinator. The response rate for principals was 65%; for 
coordinators was 70%; and overall was 68%. 

Questions were included about strategies used and barriers to implementation. See 
Appendix B for the survey questions. 

Because the initiative is so new, we used many open-ended questions to determine what 
strategies were used and what barriers to implementation principals and coordinators 
encountered. These questions were coded independently by two individuals and the codes 
were subsequently reconciled. Codes were collapsed into larger categories for ease of 
analysis. 

Focus groups 

Two focus groups were conducted at CCSR in June 2011. The first focus group had eight 
participants; the second had six. Each participant worked within schools as a vendor for 
CoC. 

In the focus groups, the vendors were asked the following kinds of questions: 

• What kind of work did you do for the school? 

• Have any of you worked with CPS in the past? 

• What was your overall impression of the Culture of Calm initiative and the work you were doing? 

• Were there specific strategies you found particularly effective? 

• What evidence do you have that they were effective? 

• Were there any strategies that seemed ineffective? 

• Would you have done anything differently? 

• How did you become involved with the schools? 

• Were there any other visible effects of the initiative in the school? 

Each focus group took about 90 minutes to complete. Each participant was provided $30 
as a token of appreciation for their time. The focus groups were audio taped. 

After each focus group, notes from two CCSR staff members were used to describe the 
experiences of these vendors in the Culture of Calm initiative. 


Survey of students and teachers (My Voice, My School) 

The surveys of students and teachers have been administered since the early 1990s. From 
2005 through 2011, all students in grades 6 through 12 as well as all teachers were asked 
to participate. 

Table 7. Response rates (total number of students in CPS) by year and grade level. 



2005 

2007 

2009 

2011 

Number of schools 
Elementary 

Grades 6+ 

484 

484 

484 

488 

Under grade 6 

68 

69 

43 

38 

High school 

123 

130 

142 

143 

Student response rates 

Elementary 

68% 

83% 

59% 

83% 


(98,681) 

(94,327) 

(89,808) 

(83,361) 

High school 

59% 

62% 

49% 

68% 


(102,129) 

(104,056) 

(106,830) 

(114,197) 

Teacher response rates 

Elementary 

60% 

74% 

52% 

48% 


(18,610) 

(17,277) 

(17,951) 

(16,158) 

High school 

54% 

63% 

54% 

49% 


(7635) 

(7510) 

(8100) 

(8096) 


All survey questionnaires can be found on CCSR’s website (ccsr.uchicago.edu). Each of the 
measures discussed in this memo were developed prior to 2005. To develop the items, 
stakeholders within CCSR and CPS were consulted. Students and teachers reviewed the 
surveys. 

We use advanced techniques to score the survey similar to those used by national 
standardized test makers. Using Rasch analysis, we combine data from a set of questions 
conceptually related to each other This technique provides us with one score for the 
concept while taking into account that some questions are more difficult to agree with than 
others. For example, teachers were asked to rate the extent to which certain behaviors are 
a problem at their school in a teacher measure called Disorder and Crime. "Threats of 
violence toward teachers” is more difficult to endorse than “disorder in hallways” because 
it likely happens far less often. 

Our scoring technique also allows us to account for the difference within questions. For 
example, it may be easier to go from strongly disagree to disagree than it is to go from 
agree to strongly agree. Rasch takes those differences into account for every question. 

For more technical details about Rasch analysis, see 

http://ccsr.uchicago.edu/downloads/9585ccsr rasch analysis primer.pdf . 


Survey administration occurs in the spring. Prior to 2011, all survey administration was 
conducted on paper, generally during the month of April. In 2011, all survey 
administration was conducted on the web. In 2011, high schools took the survey in March, 
while elementary schools took the survey in late March and early April. 



Appendix B. Survey questionnaire for web survey of principals and 
coordinators 


Introduction 

Thank you for taking the time to participate in this survey for the Culture of Calm initiative. 
We are asking people to share the strategies used by their schools to increase school safety 
so that we can learn what strategies Culture of Calm schools used, and what lessons we can 
learn from schools’ experiences with their strategies. We want to know where schools put 
their efforts, what seemed to work, and what did not. This will help us design strategies 
and programs in the future to improve the climate in CPS schools. 

This is a confidential survey, and the responses from individual schools will not be made 
publicly available. Your responses will be combined with those from other principals and 
Culture of Calm coordinators at other schools to catalogue how schools implemented the 
Culture of Calm, to identify conditions for success, and to note challenges encountered 
across the Culture of Calm schools. 

Basic information 

1 . Are you a: 

Response categories: School principal, Culture of Calm coordinator 

2. This school year, about what percent of your time during an average week did you spend on 
Culture of Calm strategies? 

[openend] 


Strategies using vendors 

Most schools in the Culture of Calm hired external agencies to help implement this 
initiative. Some of these vendors used more than one strategy. In this section, we are 
interested in the strategies employed by vendors. 

3. Please indicate if the following programs were a primary or secondary focus of the Culture of 
Calm initiative in your school. We are interested in programs that were implemented, not just 
intended to be implemented. 

• Tutoring programs 

• Mentoring programs 

• Student leadership programs 

• Professional development for teachers regarding classroom management or discipline 

• Peer mediation training for students 

• Anger management/self-regulation programs for students 


Primary, Secondary, Not a focus 


4. Please list any other strategies your vendors used. 


• First strategy [OPENEND TEXTBOX] 

• Second strategy [OPENEND TEXTBOX] 

• Third strategy [OPENEND TEXTBOX] 

5. For each of the above strategies, please indicate if it was a primary focus of the vendor(s) in your 
school. 

• First strategy 

• Second strategy 

• Third strategy 

Primary, Secondary 

6. To what extent were your vendor’s/vendors’ programs coordinated with teaching and learning at 
the school? 

Operated independently, Loosely coordinated, Somewhat coordinated, Very tightly coordinated 

7. What were some of the barriers that your vendor(s) encountered in implementing these strategies? 
(Check all that apply.) 

Lack of teacher or staff buy-in 

Lack of regular student attendance 

Lack of vendor understanding about underlying causes of safety issues at my school 
Vendor staff turnover 
Inaccurate records 

8. Please list any other barriers that your vendor(s) encountered in implementing these strategies. 
[OPENEND] 

9. Were there any strategies your vendors had intended to use but did not implement? 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 

10. Were there any strategies your vendors had intended to use but only partially implemented? 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 


11. Besides vendors, who else in your school community worked on your vendor’s/vendors’ 
strategies? Check all that apply. 



Principal , Culture of Calm coordinator, Teachers, other staff members, students, parents, the 
community, Vendor staff only 

12. Thinking about the strategies used that involved vendors, how effective did you think they were 
at increasing school safety? 

Very ineffective, Somewhat ineffective, Somewhat effective, Very effective 

13. Why do you say that your strategies were or were not effective? [OPENEND] 

Strategies not using vendors 

In addition to the services provided by your vendor(s), we'd like to know about other 
school-based strategies. This section will ask questions about other strategies your school 
may have taken as part of the Culture of Calm initiative that did not use vendors . 

14. Please indicate if the following programs that did not use vendors were a primary or secondary 
focus of the Culture of Calm initiative in your school. We are interested in programs that were 
implemented, not just intended to be implemented. 

• Changing policies or practices around attendance/tardiness 

• Changing policies or practices around discipline/suspension 

• Using incentives to help change student behavior (e.g., putting names on a wall for good 
attendance) 

• Reconfiguring staff duties 

• Redeploying security personnel 

• Changing the focus of student advisory 

• Improving record keeping procedures 

• Increased efforts at parent involvement 

Primary, Secondary, Not a focus 


15. Please list any other strategies you used (not including those provided by your vendor(s)). 

• First strategy [OPENEND TEXTBOX] 

• Second strategy [OPENEND TEXTBOX] 

• Third strategy [OPENEND TEXTBOX] 

16. For each of the above strategies, please indicate if it was a primary focus in your school. 

• First strategy 

• Second strategy 

• Third strategy 


Primary, Secondary 


17. To what extent were the strategies that did not use vendors coordinated with teaching and 
learning at the school? 

Operated independently, Loosely coordinated, Somewhat coordinated, Very tightly coordinated 


18. What were some of the barriers that you encountered in implementing these strategies? (Check all 
that apply.) 

Lack of teacher or staff buy-in 
Lack of regular student attendance 

Lack of vendor understanding about underlying causes of safety issues at my school 
Vendor staff turnover 
Inaccurate records 

19. Please list any other barriers that you encountered in implementing these strategies. [OPENEND] 

20. Were there any other strategies you had intended to use but did not implement? 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 

21. Were there any other strategies you had intended to use but only partially implemented? 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 

• [OPENEND TEXTBOX] 

22. Who in your school community worked on the strategies that did not involve vendors? (Check all 
that apply.) 

Principal, Culture of Calm coordinator, Teachers, other staff members, students, parents, the 
community, Vendor staff only 

23. Thinking about the strategies specified above that did not use vendors, how effective did you 
think they were at increasing school safety? 

Very ineffective, Somewhat ineffective, Somewhat effective, Very effective 

24. Why do you say that your strategies were or were not effective? [OPENEND] 

Culture of Calm experience as a whole 

Now we’d like to ask a couple more questions about your experience with Culture of Calm 
strategies as a whole. Please think about all of the strategies you used as part of the 
initiative, whether it involved a vendor or not. 


25. Of all of the strategies you used, which do you feel was most effective? [OPENEND] 

26. Why was this strategy the most effective? [OPENEND] 

27. Of all of the strategies you used, which do you feel was least effective? [OPENEND] 

28. Why was this strategy the least effective? 

29. Overall, to what extent do you think the Culture of Calm initiative led to an improvement in 
school climate in your school? 

No improvement , A little improvement , Some improvement , A great deal of improvement 

30. Why or why not? [OPENEND] 


Legacy of the initiative 

We’d like to know about your school’s plans for the next school year with respect to programs you 
implemented as part of the Culture of Calm initiative. 

31. In the 2011-2012 school year, is your school planning to continue using strategies that were 
implemented as part of the Culture of Calm initiative? 

Yes, no 

[IF YES TO 31:] 


32. Which strategies do you think will be implemented in the 2011-2012 school year? [OPENEND] 

33. What kinds of barriers do you think your school will face in implementing these strategies? 
[OPENEND] 


Wrap up 

Is there anything else you'd like us to know about the Culture of Calm initiative? 
[OPENEND] 


Thank you very much for your time and your feedback. 


Appendix C. Supplementary Trends in My Voice, My School Survey Data. 


As noted on page 16, a number of measures did not change substantially between 2009 and 
2011 for CoC schools compared to non-CoC schools. Below are the trend lines for each of 
these measures. 
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The following figures show the measure trends for elementary schools compared to high 
schools. 
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The following trend lines refer to differences between pilot schools and the other 41 CoC 
schools. 
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